把这个选项进到主线了https://github.com/ggml-org/llama.cpp/commit/7f323a589f8684c0eb722e7309074cb5eac0c8b5

David's random thoughts

改一下llama.cpp把bs>=32时GPU通过PCIe读取主机内存里的模型的逻辑去掉，现在内存里的tensor全部由CPU处理，速度就正常多了。无论是用8G显存还是96G显存的pp性能都各自提升到10倍，远超纯CPU运行的性能。prefill现在可以跑出300 token/s的速度，基本达到日常可用水准。

把这个选项进到主线了

https://github.com/ggml-org/llama.cpp/commit/7f323a589f8684c0eb722e7309074cb5eac0c8b5

GitHub

Add `--no-op-offload` to improve `-ot` pp perf in MoE models like lla… · ggml-org/llama.cpp@7f323a5

…ma4 400B (#13386)

www.tg-me.com/sg/David& 39;s random thoughts/com.david_random/579

2.0K viewsMay 12 at 10:09

tg-me.com/david_random/579

Create: 2025-05-12
Last Update: 2025-06-28 01:33:48

把这个选项进到主线了

https://github.com/ggml-org/llama.cpp/commit/7f323a589f8684c0eb722e7309074cb5eac0c8b5

BY David's random thoughts

Share with your friend now:
tg-me.com/david_random/579

David& 39;s random thoughts Telegram | DID YOU KNOW?

Among the actives, Ascendas REIT sank 0.64 percent, while CapitaLand Integrated Commercial Trust plummeted 1.42 percent, City Developments plunged 1.12 percent, Dairy Farm International tumbled 0.86 percent, DBS Group skidded 0.68 percent, Genting Singapore retreated 0.67 percent, Hongkong Land climbed 1.30 percent, Mapletree Commercial Trust lost 0.47 percent, Mapletree Logistics Trust tanked 0.95 percent, Oversea-Chinese Banking Corporation dropped 0.61 percent, SATS rose 0.24 percent, SembCorp Industries shed 0.54 percent, Singapore Airlines surrendered 0.79 percent, Singapore Exchange slid 0.30 percent, Singapore Press Holdings declined 1.03 percent, Singapore Technologies Engineering dipped 0.26 percent, SingTel advanced 0.81 percent, United Overseas Bank fell 0.39 percent, Wilmar International eased 0.24 percent, Yangzijiang Shipbuilding jumped 1.42 percent and Keppel Corp, Thai Beverage, CapitaLand and Comfort DelGro were unchanged.

Telegram Gives Up On Crypto Blockchain Project

Durov said on his Telegram channel today that the two and a half year blockchain and crypto project has been put to sleep. Ironically, after leaving Russia because the government wanted his encryption keys to his social media firm, Durov’s cryptocurrency idea lost steam because of a U.S. court. “The technology we created allowed for an open, free, decentralized exchange of value and ideas. TON had the potential to revolutionize how people store and transfer funds and information,” he wrote on his channel. “Unfortunately, a U.S. court stopped TON from happening.”